High-quality dense 3D point clouds with active stereo and a miniaturizable interferometric pattern projector

We have built and characterized a compact, simple and flexible 3D camera based on interferometric fringe projection and stereo reconstruction. The camera uses multi-frame active stereo as basis for 3D reconstruction, providing full-field 3D images with 3D measurement standard deviation of 0.09 mm, 12.5 Hz 3D image capture rate and 3D image resolution of 500 × 500 pixels. Interferometric projection enables a compact, low-power projector that consumes < 1 W of electrical power. The key component in the projector, a movable micromirror, has undergone initial vibration, thermal vacuum cycling (TVAC) and radiation testing, with no observed component degradation. The system's low power, small size and component longevity makes it well suitable for space applications.publishedVersio

Real-time super-resolved 3D in turbid water using a fast range-gated CMOS camera

We present a range-gated camera system designed for real-time (10 Hz) 3D estimation underwater. The system uses a fast-shutter CMOS sensor (1280×1024 1280×1024 ) customized to facilitate gating with 1.67 ns (18.8 cm in water) delay steps relative to the triggering of a solid-state actively Q -switched 532 nm laser. A depth estimation algorithm has been carefully designed to handle the effects of light scattering in water, i.e., forward and backward scattering. The raw range-gated signal is carefully filtered to reduce noise while preserving the signal even in the presence of unwanted backscatter. The resulting signal is proportional to the number of photons that are reflected during a small time unit (range), and objects will show up as peaks in the filtered signal. We present a peak-finding algorithm that is robust to unwanted forward scatter peaks and at the same time can pick out distant peaks that are barely higher than peaks caused by sensor and intensity noise. Super-resolution is achieved by fitting a parabola around the peak, which we show can provide depth precision below 1 cm at high signal levels. We show depth estimation results when scanning a range of 8 m (typically 1–9 m) at 10 Hz. The results are dependent on the water quality. We are capable of estimating depth at distances of over 4.5 attenuation lengths when imaging high albedo targets at low attenuation lengths, and we achieve a depth resolution () (σ)ranging from 0.8 to 9 cm, depending on signal level.publishedVersio

Autonomous subsea intervention (SEAVENTION)

This paper presents the main results and latest developments in a 4-year project called autonomous subsea intervention (SEAVENTION). In the project we have developed new methods for autonomous inspection, maintenance and repair (IMR) in subsea oil and gas operations with Unmanned Underwater Vehicles (UUVs). The results are also relevant for offshore wind, aquaculture and other industries. We discuss the trends and status for UUV-based IMR in the oil and gas industry and provide an overview of the state of the art in intervention with UUVs. We also present a 3-level taxonomy for UUV autonomy: mission-level, task-level and vehicle-level. To achieve robust 6D underwater pose estimation of objects for UUV intervention, we have developed marker-less approaches with input from 2D and 3D cameras, as well as marker-based approaches with associated uncertainty. We have carried out experiments with varying turbidity to evaluate full 6D pose estimates in challenging conditions. We have also devised a sensor autocalibration method for UUV localization. For intervention, we have developed methods for autonomous underwater grasping and a novel vision-based distance estimator. For high-level task planning, we have evaluated two frameworks for automated planning and acting (AI planning). We have implemented AI planning for subsea inspection scenarios which have been analyzed and formulated in collaboration with the industry partners. One of the frameworks, called T-REX demonstrates a reactive behavior to the dynamic and potentially uncertain nature of subsea operations. We have also presented an architecture for comparing and choosing between mission plans when new mission goals are introduced.publishedVersio

Bruk av droner i nordområdene

Formålet med denne rapporten er å vise eksempler på bruk av fjernstyrte og autonome droner i petroleumssektoren og vise hvilke utviklingstrender innen droneteknologi som er viktige. Utfordringer og muligheter for bruk av droner i nordområdene er spesielt vektlagt. Teknologi for og anvendelser av flyvende droner, droner på havoverflaten og droner under vann er behandlet i denne rapporten. Rapporten beskriver hvordan droner brukes i dag og oppgaver dronene kan løse fremover. Rapporten beskriver utfordringer ved bruk av droner innen petroleumsvirksomheten og forslag til tiltak

Design tool for TOF and SL based 3D cameras

Active illumination 3D imaging systems based on Time-of-flight (TOF) and Structured Light (SL) projection are in rapid development, and are constantly finding new areas of application. In this paper, we present a theoretical design tool that allows prediction of 3D imaging precision. Theoretical expressions are developed for both TOF and SL imaging systems. The expressions contain only physically measurable parameters and no fitting parameters. We perform 3D measurements with both TOF and SL imaging systems, showing excellent agreement between theoretical and measured distance precision. The theoretical framework can be a powerful 3D imaging design tool, as it allows for prediction of 3D measurement precision already in the design phase.publishedVersio

Multimodal Image Registration for Preoperative Planning and Image-Guided Neurosurgical Procedures

Image registration is the process of transforming images acquired at different time points, or with different imaging modalities, into the same coordinate system. It is an essential part of any neurosurgical planning and navigation system because it facilitates combining images with important complementary, structural, and functional information to improve the information based on which a surgeon makes critical decisions. Brigham and Women's Hospital (BWH) has been one of the pioneers in developing intraoperative registration methods for aligning preoperative and intraoperative images of the brain. This article presents an overview of intraoperative registration and highlights some recent developments at BWH

Adaptive Structured Light with Scatter Correction for High-Precision Underwater 3D Measurements

High-precision underwater 3D cameras are required to automate many of the traditional subsea inspection, maintenance and repair (IMR) operations. In this paper we introduce a novel multi-frequency phase stepping (structured light) method for high-precision 3D estimation even in turbid water. We introduce an adaptive phase-unwrapping procedure which uses the phase-uncertainty to determine the highest frequency that can be reliably unwrapped. Light scattering adversely affects the phase estimate. We propose to remove the effect of forward scatter with an unsharp filter and a model-based method to remove the backscatter effect. Tests in varying turbidity show that the scatter correction removes the adverse effect of scatter on the phase estimates. The adaptive frequency unwrapping with scatter correction results in images with higher accuracy and precision and less phase unwrap errors than the Gray-Code Phase Stepping (GCPS) approach

SEAVENTION - Autonomous underwater intervention

submittedVersio

6D pose estimation for subsea intervention in turbid waters

Manipulation tasks on subsea instalments require extremely precise detection and localization of objects of interest. This problem is referred to as “pose estimation”. In this work, we present a framework for detecting and predicting 6DoF pose for relevant objects (fish-tail, gauges, and valves) on a subsea panel under varying water turbidity. A deep learning model that takes 3D vision data as an input is developed, providing a more robust 6D pose estimate. Compared to the 2D vision deep learning model, the proposed method reduces rotation and translation prediction error by (−Δ0.39∘) and translation (−Δ6.5 mm), respectively, in high turbid waters. The proposed approach is able to provide object detection as well as 6D pose estimation with an average precision of 91%. The 6D pose estimation results show 2.59∘ and 6.49 cm total average deviation in rotation and translation as compared to the ground truth data on varying unseen turbidity levels. Furthermore, our approach runs at over 16 frames per second and does not require pose refinement steps. Finally, to facilitate the training of such model we also collected and automatically annotated a new underwater 6D pose estimation dataset spanning seven levels of turbidity